An Efficient Protocol Specification, Implementation, and Evaluation for a Highly Scalable Peer-to-Peer Search Infrastructure

While scalable mechanisms for lookup of unique IDs in peer-to-peer (P2P) systems have been found, scalability remains an issue for P2P keyword search. Therefore, a new solution, the SHARK algorithm, has been proposed. Constructing a symmetric redundant hierarchy of nodes and information objects allows for efficient query routing toward small semantic clusters of peers. To show this algorithm’s applicability, a detailed specification of the SHARK protocol and a thorough evaluation of its performance is provided. In addition to proving the validity and technical feasibility of the algorithm, this forms the basis for large scale use in several P2P applications. While providing rich keyword search functionality, it is shown that SHARK can easily achieve four orders of magnitude scalability improvement over Gnutellalike networks, greatly outperforming approaches like expanding ring search or associative overlays.